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Abstract 

Several important classes of images such as text, barcode 
and pattern images have the property that pixels can only 
take a distinct subset of values. This knowledge can benefit 
the restoration of such images, but it has not been widely 
considered in current restoration methods. In this work, we 
describe an effective and efficient approach to incorporate 
the knowledge of distinct pixel values of the pristine images 
into the general regularized least squares restoration frame¬ 
work. We introduce a new regularizer that attains zero at the 
designated pixel values and becomes a quadratic penalty 
function in the intervals between them. When incorporated 
into the regularized least squares restoration framework, 
this regularizer leads to a simple and efficient step that re¬ 
sembles and extends the rounding operation, which we term 
as soft-rounding. We apply the soft-rounding enhanced so¬ 
lution to the restoration of binary text/barcode images and 
pattern images with multiple distinct pixel values. Exper¬ 
imental results show that soft-rounding enhanced restora¬ 
tion methods achieve significant improvement in both visual 
quality and quantitative measures (PSNR and SSIM). Fur¬ 
thermore, we show that this regularizer can also benefit the 
restoration of general natural images. 

1. Introduction 

The task of image restoration is to recover an image from 
its noisy and/or blurry observation [3]. Image restoration is 
an example of ill-posed inverse problems, and unique solu¬ 
tions can only be obtained by introducing proper regulariza¬ 
tions and constraints on the clean images. Indeed, it is now 
generally accepted that the more prior knowledge we have 
about the properties of the uncorrupted images, the better 
we can solve the image restoration problem [1,9]. 

To date, the most widely used statistical regularities for 
images are obtained from their band-pass filter responses 
(such as gradients or wavelets) or mean-removed pixel 
patches. Such responses have been observed to have heavy¬ 


tailed marginal histograms, on the basis of which regular- 
izers or priors that prefer sparse responses have been used to 
attain the state-of-the-art restoration performances. Exam¬ 
ples include regularizers in the form of the norm [25, 26] 
and total variation [20], or prior models based on general¬ 
ized Laplacian [11, 13] and Gaussian mixtures [23, 28]. In 
these image restoration methods, pixel values are usually 
treated as continuous variables. 

However, images are captured and stored in a digital for¬ 
mat, which poses a general constraint on pixel values. The 
pixels of an image in a 6-bit format can only take integer 
values from the set {0, • • • ,2^ — 1}. For some important 
subclasses of images such as text or pattern images, their 
pixels can take only a distinct set of the full range integer 
values (see Fig. 1(a) for an example). Furthermore, it is usu¬ 
ally possible to obtain these values before we restore such 
an image. For text and barcode images, the pixel values 
are usually separated into two distinct classes of foreground 
pixels and background pixels. For pattern images, image 
pixels usually take multiple distinct pixel values, which can 
be estimated from other images of the same type. 

Knowing that the pristine image can only take some dis¬ 
tinct pixel values provides a useful constraint for image 
restoration, which can help to suppress visual artifacts and 
improve the restoration performance. A straightforward ap¬ 
proach would be to round the restored pixel values to the 
nearest distinct pixel values as a post-processing step. How¬ 
ever, such a simple solution often does more harm than 
good, as the rounding step can undo the structures recovered 
from the restoration step (see Fig. 1(c)). On the other hand, 
except in a few cases [10, 16, 18, 27], such prior information 
about pixel values seems trivial and is largely overlooked by 
most existing image restoration algorithms. 

In this work, we describe an effective and efficient ap¬ 
proach to incorporate the knowledge of distinct pixel values 
of the pristine images into the popular regularized least 
squares restoration framework. We introduce a new regu¬ 
larizer of the uncorrupted images in the pixel domain. This 
regularizer attains zero at the designated pixel values and is 
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(a) original image (b) corrupted image (c) restoration [18] (d) + post rounding (e) + soft-rounding 

PSNR = 19.45dB PSNR = 19.28dB PSNR = 19.84dB 

SSIM = 0.79 SSIM = 0.82 SSIM = 0.85 

Figure 1: An example demonstrating the ejfect of our method, (a) An original image with four distinct pixel values 0,100,150, 255. (b) 
Its corruption with a 101 x 101 blur kernel and 1% white Gaussian noise, (c) Restoration results using the state-of-the-art method based 
on Iq regularization of image gradients [18]. The artifacts in the restored image are due to pixel values that are not consistent with the 
four distinct pixel values, (d) Rounding (nearest neighbor) of the restoration results in (c) to the four distinct pixel values. Though some 
artifacts in (c) are removed, some structures restored in (c) are also destroyed, (e) Restoration results using our method that combines 
gradient domain Iq regularization based restoration with soft-rounding operation. Note that our method achieves significant improvement 
in comparison with (c) and (d) both visually and quantitatively (PSNR and SSIM). 


a quadratic function in the intervals between them. As such, 
it reflects the preference of the pixel values of the restored 
image to take the designated distinct pixel values. When 
incorporated into the regularized least squares restoration 
framework, this regularizer leads to a simple and efficient 
solution that resembles the rounding operation, which we 
term as soft-rounding. We apply the soft-rounding en¬ 
hanced solution to various state-of-the-art image restoration 
methods, and show that it achieves significant improvement 
in both visual quality and quantitative measures (PSNR and 
SSIM) for the restoration of binary images and multi-value 
pattern images. We also demonstrate its effectiveness for 
general natural image restoration. 

To sum up, the main contributions of this work are: 

• we introduce a new regularizer encouraging the output 
image to take designated distinct pixel values; 

• unlike previous binary image restoration methods [18, 
27], our regularizer can intrinsically handle more than 
two distinct pixel values; 

• optimization with this regularizer leads to a simple and 
efficient soft-rounding operation that allows easy inte¬ 
gration with existing restoration algorithms; 

• when applied to the restoration of text and pattern im¬ 
ages, our method achieves significant improvement in 
both quantitative and visual quality. 

2. Background and Related Work 

We formulate the image restoration problem as recover¬ 
ing a pristine image of m pixels, which is vectorized and 
denoted with an m-dimensional vector x, from its noisy 
and/or blurry observation y. We assume that y is generated 
from the convolution of x with a spatially invariant blurring 
kernel k and further contaminated with additive Gaussian 


noise n, as: 

y = k(g)x + n, (1) 

where (g) denotes the convolution operation [3]^ The 
restoration problem can be solved within a regularized least 
squares (RLS) estimation framework, as: 

min^||y--fi^x||2 + Aivrjv(-Dx) (2) 

X 2 

where K corresponds to the block Toeplitz matrix repre¬ 
sentation of the convolution kernel k, D represents a lin¬ 
ear transform through which image properties can be better 
modeled. Tat is a regularizer in the transformed domain, 
and Aat is a adjustable parameter balancing the contribution 
of the data fidelity term and the regularization term in the 
objective function. For image denoising, K reduces to an 
identity matrix. 

The choice of the linear transform D and the regular¬ 
izer Fat is essential for effective restoration. A common 
approach is to first transform image to band-pass domains 
(e.g., gradients or wavelets) or use mean-removed patches, 
then use regularizers in the form of sparsity-encouraging 
norms, e.g., Iq [25, 26] and total variation [20], or models 
capturing statistical properties of images in such domains, 
e.g., generalized Laplacian [11, 13], Gaussian scale mix¬ 
tures [23], or general Gaussian mixtures [28]. 

To date, properties of the pristine images in the pixel 
domain have been largely overlooked in image restoration 
but in a few cases [4, 10, 16, 18, 27]. The constraint on 
pixels to take distinct values has been considered in a few 
recent works in binary image restoration, where the pixels 
can take only two values. Zhang introduced a regularization 

Tn the current work, we consider the setting that the blurring kernel and 
noise variance are known, known as non-blind restoration. Our method can 
also be extended to blind restoration [14]. 



term to penalize pixel values that drift away from the two 
values [27]. The specific regularizer used there is a fourth- 
order polynomial and requires iterative numerical approx¬ 
imations on each pixel. Pan et al. used a regulariza¬ 
tion term if only the zero peak in pixel values is consid¬ 
ered [18]. They showed that this intensity prior helps to 
identify salient edges in text images and therefore improves 
blur kernel estimation, but its non-blind restoration perfor¬ 
mance was not reported in the paper. A potential limitation 
of the ^0 intensity prior is that it might not be suitable for 
binary images whose pixel values center around two peaks 
far away from zero. Another common limitation of these bi¬ 
nary image restoration methods is that they can not be easily 
extended to handle more than two distinct pixel values. 

Some recent methods further incorporate histograms of 
the pixel values in image restoration. Chen et al. stud¬ 
ied the intensity histograms of clear document images and 
used them as reference information for blind deconvolu¬ 
tion, leading to improved restored results with reduced vis¬ 
ual artifacts [4]. Mei et al. recently proposed a framework 
to enforce marginal histogram constraints in image restora¬ 
tion. They showed that for pattern images, marginal inten¬ 
sity histograms can significantly improve the denoising per¬ 
formance under high noisy levels [16]. Although a pixel 
histogram provides more information than specifying dis¬ 
tinct pixel values in an image, estimating such histograms 
from degraded images is a non-trivial task [5, 16, 29], which 
poses a limitation for these methods in practice. In con¬ 
trast, as we pointed out in Section 1, obtaining distinct pixel 
values before restoring an image is usually possible for the 
type of images we focus on in this work. 

3. Method 

As discussed previously, some subclasses of images have 
distinct pixel values. When these values are known a priori, 
they provide valuable information to improve the restora¬ 
tion performance. In this section, we describe in detail an 
approach to incorporate such information into the general 
RLS restoration framework, Eq.(2). It turns out that the spe¬ 
cific solution we obtained has a particularly intuitive inter¬ 
pretation as a “soft-rounding” operation, which adaptively 
adjust the restored pixel values to the target values. 

3.1. Distinct Pixel Value Regularizer 



Figure 2: An example of ^t{x) in the range of [0,1], with five 
distinct values (o) corresponding to [0.1,0.5, 0.7, 0.8, 0.9]. 

where m is the number of the image pixels, and 7 t measures 
the intensity deviation on each pixel: 


lt{x) 


— x) if X < ti 

< ^{x - - x) if X G [tj,tj+i] . (4) 

\{x-tn) if X>tn 


Fig.2 shows an example of 7t(x) in the range of [0,1]. 
This function has the following distinct characteristics: 

1- 7t(^) > 0 for X G 7^; 

2. ^t{x) attains zero at tj {j = 1, • • • ,n) and reaches 
local maximum when x locates at the midpoint of each 
interval; 

3. for X G 7t(x) is a concave quadratic func¬ 

tion, and 7t (x) reduces to a linear function outside the 
range of 

Note that 7 t (x) is neither a convex nor a concave function 
over the whole real line, but at individual intervals it is con¬ 
cave (it becomes either a quadratic or linear function). 


3.2. Using T/ in Image Restoration 

We incorporate regularizer r/(x) into the RLS restora¬ 
tion framework and solve the following problem: 

minl||y - Kx\\l + \nTn{Dx.) + XiTi{x) (5) 

X 2 

where we introduce two parameters A at , A/ to balance the 
contributions of the three terms in the overall objective 
function. Note that Eq.(5) is a non-convex problem, and 
we optimize it for a local minimum. 

Following the general variable splitting scheme [2], we 
introduce an auxiliary variable z/ and convert Eq.(5) to the 
following equivalent problem: 

min 4||y - Kx||| + Ajvriv(r>x) + A 7 r/(z/) 

X (6) 

S.t. Z/ = X 


We first introduce a regularizer to encourage pixel values 
of the restored image to take the designated distinct pixel 
values. Formally, assuming we require the restored image 
should have n distinct pixel values ti < ^2 * * * < tn^ repre¬ 
sented in a vector t = (ti, ^2, * * * ? we define the regu¬ 
larizer as 

(3) 


We then solve this equality constrained problem using the 
augmented Lagrangian method (ALM) [17]. Introducing 
a Lagrange multiplier w/ to the equality constraint and a 
penalty parameter pj, we form the augmented Lagrangian 
function of problem (6), as 

£(x,z/,w/) = A||y-ii:x||| + Aivrjv(-Dx) 
+A/r/(z/) + 7||x - z/lll - wf(x - z/) 


r/(x) = Er=i7t(:r,) 


(7) 






Starting with initial values for z/, x, w/ and parameter /ij, 
we iteratively update z/,x,w/ with regards to the aug¬ 
mented Lagrangian starting from /c = 1 as follows: 

1. Update z/! zj^^ = argmin^^ >C(x^,z/, wj) 

2. Update x: x^+^ = argmin^ >C(x, zj+^, Wj) 

3. Update w/: = Wj — /i/(x^+^ — 

It is guaranteed in theory that the solution to the original 
problem (6) can be obtained by solving (7) iteratively in 
the ALM framework when penalty parameter jij increases 
to a sufficiently large value [17]. Compared with the half¬ 
quadratic splitting method [8, 1 1], a significant advantage of 
ALM is that its convergence can be assured without increas¬ 
ing /i/ indefinitely, which is especially suitable for solving 
problems with multiple equality constraints. 

The update step for the multiplier (step 3) is straightfor¬ 
ward as required by the ALM framework [17]. The sub¬ 
problem updating x (step 2) can be reformulated as a RLS 
problem: 


mini ||y-ii:x||2 + ^ 




/i/ 


n{D^) 


( 8 ) 

Compare to Eq.(2), Eq.(8) has an additional quadratic term 
about X, i.e., the second term. If Tat is also a quadratic term 
on X, this subproblem can be solved through conjugate gra¬ 
dient or EET-based method. Eor most existing restoration 
methods with non-smooth regularizers, this subprob¬ 
lem can be further decomposed into smaller subproblems 
through half-quadratic splitting or ALM [8, 11], for which 
well-developed proximal operators for existing T ^ regular¬ 
izers can be reused in the computation process. Note that 
for practical implementations, the decomposition of the x 
subproblem can be done within the same ALM framework 
as z/, and its solution requires only a one-iteration approxi¬ 
mate updating process. Compared to standard restoration 
methods with only Tat terms, our method requires extra 
slight computation costs on the z/ and w/ updating steps. 
Eor better understanding of the x subproblem, please refer 
to the supplemental material for two concrete examples in¬ 
volving different Tat terms. The solution of the z/ (step 1) 
is detailed in the following section. 


3.3. Soft-Rounding Operator 


After rearranging terms, the subproblem of updating z/ 
(step 1) reduces to the following minimization problem: 

||z/-c|| 2 +A/r/(z/), (9) 

Z/ Z 

where we define c = x^-Note that both terms 

Ml i 

in Eq.(9) can be split into the the sum on each element of 
z/ (see the definition of r/(-) in Eq.(3)). Therefore this 
problem can be optimized separately on each element as 

(zj+^)i = A ^(Ci), (10) 



Eigure 3: Examples of function ft.xic) for different A values 
and the same set of t — [0.1,0.5,0.7,0.8,0.85] (o). Left: soft- 
rounding (\ — 0.6A Right: rounding (\ — 1.1). 

where 


4>t,\{c) = argmin 

X 




( 11 ) 


can be seen as the proximal operator of function 7 t(') [19]. 

The following result shows that 0t,A (c) can be computed 
with a piece-wise function. 

Theorem 1. For constant A > 0 and ti < ^2 * * * < ihe 
optimal solution to Eq.{ll) is given by: 

(i) forc < ti, (/>t,A(c) = inin(ti,c+ |); 

(ii) for c > tn, = max(t„, c - |); 

(Hi) hard-rounding: for c G [ti, tn] (^nd A > 1, 

, 12 ) 

where dj = |(t^+i — tj) and j = 1, • • • , n — 1; 

(iv) soft-rounding: for c G [ti, tn] and A G (0,1), 


0t,A(c) = < 


1-A 
^j + 1 


c G [tj,tj + dj] 

Hh+h+i) ^ ^ ^-f-. dj.tj^i - dj] 


2(1-A) 


^ ^ fe+i 5 ^i+i] 


where dj = f (tj+i — tj) and j = 1, • • • , n — 1. 


(13) 


The proof of Theorem 1 is given in the supplementary 
material. In Eig.3 we show examples of ft,x with different 
A values and the same set of t. As can be seen from Theo¬ 
rem 1 and Eig.3, when A > 1, 0t,A maps c G [tj,tj+i] to 
the nearest of tj or tj^i,i.e., it rounds c to either tj or t^+i. 
Eor 0 < A < 1, ft, X keeps a zone [tj + dj^tj^i — dj] C 
[tj,tj+i], inside which c is not rounded but undergoes a 
linear transform, the slope of which is determined by A. As 
such, we term Eq.(13) as soft-rounding, drawing an analogy 
from soft-thresholding, which is the proximal operator of 
norm [7]. The soft-rounding operator “softens” the round¬ 
ing operator, and has an interesting effect on its output: if c 
is close to the endpoints of an interval, it will be rounded to 
the nearest endpoint. However, if c lies in the middle range 












of an interval, it is “nudged” towards the nearest t value 
through a linear interpolation. 

Note that through the ALM framework, regularizer F/ 
and the resulting soft-rounding operator can readily work 
with most existing RLS image restoration methods. The 
two regularizers, and F/, collaborate in a closed loop to 
recover the uncorrupted image. The former restores struc¬ 
tures (edges and contours), and the latter encourages the 
restored image to have the desirable distinct pixel values. 
Also, typical restoration methods tend to smooth the image, 
and the soft-rounding operator, by enhancing the contrasts 
of the restored image, serves as a countermeasure to such 
over-smoothing. 

Our method is advantageous to the simpler approach that 
first restores the image and then applies rounding to project 
the pixel values back to the target values. Since rounding 
is performed independent of and after the restoration step, 
it may destroy the structures recovered by restoration and 
undo its effect. This difference is demonstrated in the sub¬ 
sequent experiments in Section 4. 

3.4. Obtaining Distinct Pixel Values 

We first discuss several general ways to obtain distinct 
pixel values to be used in F/, depending on the applica¬ 
tion scenario and domain knowledge. (1) Distinct pixel 
values can be extracted from external sources. For instance, 
text images captured under similar illumination conditions 
might vary in spatial layout and visual contents, but they 
show very close distinct values for text and background 
pixels respectively. (2) Distinct pixel values can be spec¬ 
ified through user interaction. We first recover a clean im¬ 
age from the degraded observation with existing restoration 
methods, and then manually select dominant pixel values 
from various regions as distinct pixel values. (3) For spe¬ 
cific problems, distinct pixel values can be estimated from 
the degraded observation. For images contaminated with 
small Gaussian noise, their marginal intensity histograms 
might be roughly estimated from the noisy observations fol¬ 
lowing a ID deconvolution approach [29], from which we 
can extract distinct pixel values. 

Apart from general discussions, we also provide a simple 
but effective method to estimate distinct pixel values from 
a degraded image. A natural idea is to first recover a clean 
image with standard image restoration methods (e.g., LO) 
and then use clustering methods such as K-Means to extract 
the cluster centers as distinct pixel values. However, the im¬ 
age degradation process blurs across different distinct pixel 
values and shrinks the distances between them. This effect 
might still be evident in the restored image, sometimes lead¬ 
ing to inaccurate centers as shown Table 1 . Inspired by the 
local two-color model [10] and its recent extension [12], 
we propose a simple two-step estimation method. In the 
first step, we extract a small patch around each pixel and 


Method 

Examples 

Fig. 1 

Fig. 4 

Fig. 6 (2nd Row) 

True Values 

0,100,150,255 

26,217 

24,53,158,224,255 

K-Means 

18,98,156,245 

62,215 

14,55,164,221,253 

Our Method 

9,101,155,255 

32,217 

20,52,162,224,255 


Table 1: The distinct pixel values estimated by K-Means and our 
method for three LO deconvolution visual examples. 


estimate a local one-center or two-center model. If the in¬ 
tensity variance in the patch is below a threshold, the patch 
falls into the one-center model, and its mean pixel value is 
collected for the second global clustering step; otherwise 
the patch is a two-center patch, and its two cluster centers 
are collected with a normalized two-class K-Means cluster¬ 
ing method [12]. Compared to K-Means, normalized K- 
Means tries to stretch the distance between the two cluster 
centers and fight against the shrinkage effect due to image 
degradation. Then in the second step, we perform a global 
K-Median clustering on all the collected local center values 
to determine the final distinct pixel values. We provide the 
estimation results for three LO deconvolution text and pat¬ 
tern image examples in Table 1 : the pixel values estimated 
by our method are very close to those of the clean images 
and are more accurate than the results produced by a single 
global clustering method such as K-Means. 

4. Experiment 

In this section, we combine the soft-rounding operator 
with several state-of-the-art RLS image restoration methods 
and evaluate its performance on a variety of images. We 
focus on the restoration of three typical categories of im¬ 
ages with distinct pixel values, namely, text images with 
two pixel values, pattern images with multiple pixel values, 
and natural images with pixels of full range of 8-bit depth. 
Furthermore, we also test our algorithm on other types of 
images that are of practical importance, such as barcodes, 
posters and license plates. In all experiments, we assume 
the distinct pixel values are known. 

4.1. Text Image Deconvolution 

Text images form an important but special class of im¬ 
ages. Many text images originate from text documents and 
have two distinct pixel values for text and background pixels 
respectively. State-of-the-art non-blind restoration perfor¬ 
mance for text images was achieved with an RLS method 
with an regularizer in the gradient domain (subsequently 
denoted as LO in Table 2) [18, 25, 26]. Assuming the pixel 
values for foreground and background pixels have been ex¬ 
tracted from similar text images, we augment the -based 
RLS restoration method with soft-rounding and denoted it 
as LO-fS. We also compare the performance of LO combined 
with three other pixel domain regularizers: a simple post¬ 
processing method which directly rounds the restored im¬ 
age to a binary image (denoted as LO-fR), the fourth-order 
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(d) LO+L (PSNR = 21.01dB, SSIM = 0.87) (e) LO+P (PSNR = 21.72dB, SSIM = 0.90) (f) LO+S (PSNR = 21.88dB, SSIM = 0.92) 

Figure 4: Text Image Deconvolution Example. The original image is degraded with a 51 x 51 kernel and 1% Gaussian noise. The values 
for text and background pixels are ti = 26, t 2 = 217 respectively. Details are best viewed on screen. 


polynomial regularizer to incorporate binary intensity con¬ 
straints [27] (denoted as LO-fP) and the Iq intensity regu¬ 
larizer from [18] (denoted as LO-fL). For fair comparison 
of each method, we adjust parameters and report the best 
overall performance. 

We collect 20 text images of different languages, equa¬ 
tions and graphs. We use three kernel/noise settings to sim¬ 
ulate the degradation process: (i) a 33 x 33 kernel and 
additive Gaussian noise with 3% standard deviation; (ii) a 
45 X 45 kernel and 2% Gaussian noise; and (iii) a 51 x 51 
kernel and 1% Gaussian noise. The kernels are motion-blur 
kernels previously used in [13, 18]. 

Table 2 reports the average peak-signal-to-noise (PSNR) 
and structural similarity index (SSIM) [24] results of the 
five methods in three kernel-noise settings. Compared to 
the original LO method, the soft-rounding enhanced restora¬ 
tion method significantly improves the average PSNR by 
at least 0.5dB and the average SSIM by at least 0.04 for 
all the settings, which is more effective than the other two 
methods with intensity regularizers (LO-fL, LO-fP). The sim¬ 
ple rounding method (LO-fR) actually lowers the perfor¬ 
mance for most settings. These quantitative results demon¬ 
strate the effectiveness of our method. 


Kernel/Noise 

Avg. PSNR (in dB) 

LO 

LO+R 

LO+L 

LO+P 

LO+S 

33 X 33+3% 

19.46 

16.75 

19.71 

19.91 

20.03 

45 X 45+2% 

18.20 

15.71 

18.47 

18.57 

18.86 

51 X 51+1% 

18.03 

15.95 

18.23 

18.93 

19.16 

Kernel/Noise 

Avg. SSIM 

LO 

LO+R 

LO+L 

LO+P 

LO+S 

33 X 33+3% 

0.86 

0.84 

0.87 

0.88 

0.90 

45 X 45+2% 

0.82 

0.81 

0.83 

0.86 

0.88 

51 X 51+1% 

0.78 

0.83 

0.79 

0.83 

0.86 


Table 2: The average PSNR and SSIM results of the 20 text im¬ 
ages in three kernel-noise settings. 

but it also destroys recovered structures, which lowers both 
visual quality and quantitative measures. Using the io reg¬ 
ularizer in pixel domain (LO-fL) slightly improves over the 
LO method in PSNR and SSIM, but visual artifacts are still 
visible in the restored image. On the other hand, regular¬ 
izers specifically enforcing distinct pixel values including 
the method of [27] (LO-fP) and our regularizer (LO-fS) can 
more effectively incorporate the binary intensity constraint. 
Our method recovers more structures and suppresses more 
intensity artifacts with the soft-rounding operator, leading 
to the best visual quality and quantitative results. More ex¬ 
amples can be found in the supplemental material. 


We provide a visual example of kernel/noise setting (iii) 
in Figure 4, which contains an equation and some English 
text. As shown by the example, the original LO method pro¬ 
duces pixel values that do not exist in the original text im¬ 
age, leading to many visible artifacts. A direct rounding to 
the binary image (LO-fR) removes out-of-range pixel values. 


4.2. Pattern Image Restoration 

Pattern images are an important class of images which 
can be frequently found from human-made objects, artis¬ 
tic designs and paintings. An important property of pattern 
images is that their visual contents are represented with a 












































































































































(a) original image (b) noisy observation (c) BM3D (d) BM3D+R (e) BM3D+S 

PSNR = 19.97dB PSNR = 19.82dB PSNR = 20.93dB 

SSIM = 0.69 SSIM = 0.71 SSIM = 0.76 



(a) original image (b) noisy observation (c) BM3D (d) BM3D+R (e) BM3D+S 

PSNR = 29.65dB PSNR = 32.02dB PSNR = 34.35dB 

SSIM = 0.93 SSIM = 0.94 SSIM = 0.96 


Figure 5: Pattern Image Denoising Examples. Top row: this pattern image is degraded with 25% Gaussian noise. The distinct pixel 
values are 32, 76,142, 230 respectively. Bottom row: this pattern image is degraded with 20% Gaussian noise. The distinct pixel values 
are 32, 70,135,158, 231 respectively. Details are best viewed on screen. 


Noise 

Avg. PSNR (in dB) 

BM3D 

BM3D+R 

BM3D+S 

15% 

29.77 

29.75 

31.13 

20% 

26.71 

27.21 

28.67 

25% 

24.59 

25.21 

25.96 

Noise 

Avg. SSIM 

BM3D 

BM3D+R 

BM3D+S 

15% 

0.92 

0.91 

0.94 

20% 

0.89 

0.89 

0.92 

25% 

0.86 

0.85 

0.88 


Table 3: The average PSNR and SSIM results of the 10 pattern 
images in three noise settings. 

few distinct pixel values [16]. We test our method to re¬ 
move noise from pattern images. In our experiments, we 
collect 10 pattern image examples and degrade them with 
three high Gaussian noise levels (15%, 20% and 25%), 
such that the restoration performance depends heavily on 
the regularizers. We adopt the BM3D denoiser as the base¬ 
line method [6] and combine it with the soft-rounding reg- 
ularizer (denoted as BM3D-FS). The numerical solution of 
BM3D-FS can be found in the supplemental material. For 
comparison, we include a post-processing method which di¬ 
rectly rounds the BM3D denoised image pixels to nearest 
known pixel values (denoted as BM3D-I-R). Note that previ¬ 
ous binary intensity regularizers [18, 27] can not be easily 
extended to handle more than two distinct pixel values. 


Table 3 reports the average PSNR and SSIM results 
over 10 pattern images for the three noise levels. Com¬ 
pared to BM3D, our method (BM3D-FS) improves the av¬ 
erage PSNR by at least 1.3dB and the average SSIM by at 
least 0.02 for the three cases. The simple rounding method 
(BM3D-I-R) sometimes leads to a deterioration either in 
PSNR or in SSIM. We provide two visual examples with 
different noise levels are in Figure 5. As these results show, 
the original BM3D method tends to introduce out-of-the- 
range pixel values and blurred boundaries. The post round¬ 
ing method (BM3D-I-R) improves the image contrast with 
known intensity information, but it also introduces visible 
errors on region boundaries. On the other hand, our method 
improves both visual quality and quantitative results with 
better contrast and image structures. 

We further present several deconvolution examples of 
practical pattern images in Figure 6. We use the LO decon¬ 
volution method as the baseline method and combine the 
distinct pixel value information with the LO-fR and LO-fS 
method. As in the case of denoising, our method (LO-fS) 
significantly improves the visual quality and the quantita¬ 
tive results, while the direct rounding method (LO-i-R) in¬ 
troduces extra artifacts and removes useful structures in the 
original images. Please refer to the supplemental material 
for more examples. 









































(a) original image (b) degraded image (c) LO (d) LO+R (e) LO+S 

PSNR = 13.18dB PSNR = 12.50dB PSNR = 21.02dB 

SSIM = 0.70 SSIM = 0.80 SSIM = 0.97 





(a) original image (b) degraded image (c) LO (d) LO+R (e) LO+S 

PSNR = 18.69dB PSNR = 17.87dB PSNR = 19.13dB 

SSIM = 0.67 SSIM = 0.72 SSIM = 0.78 


Figure 6: More LO Deconvolution Examples. Top row: the ID barcode image is degraded with a 65 x 65 kernel and 1% Gaussian noise. 
The distinct pixel values for barcode and background pixels are ti = 0,t2 = 255 respectively. Bottom row: the license plate image is 
degraded with a 55 x 55 kernel and 3% Gaussian noise. The distinct pixel values are 24, 53,158, 224, 255 respectively. Details are best 
viewed on screen. 


4.3. Natural Image Deconvolution 

We randomly select 200 natural images from the 
BSDS500 dataset [15] and test them with the same ker¬ 
nel/noise settings used in Sec. 4.1. For natural image de- 
convolution, we choose the total variation method (denoted 
as TV) [20] as our baseline method. We integrate the in¬ 
teger intensity constraint into the TV method using soft- 
rounding (denoted as TV-fS) and compare it to the post¬ 
rounding method (denoted as TV-fR). 

Our experiments show that compared to TV, TV-fS 
improves the average PSNR by O.OOSdB, O.OOSdB and 
0.025dB for the three cases, respectively, while TV-i-R 
slightly degrades the performance by about 0.002dB for 
each case. Even though the average contribution of the con¬ 
straint on integer pixel values seems marginal, a detailed 
performance breakdown can shed more light on its effect. 
Specifically, we report the PSNR difference between TV-i-S 
and TV on the 200 images in Table 4: for all the three ker¬ 
nel/noise settings, TV-fS improves TV on most images; and 
for at least 11% of the images, the improvement obtained 
with TV-fS over TV is at least O.OldB; on the other hand, 
it decreases TV performance significantly (> O.OldB) only 
on one image example from all the cases. 

Summary. As all the experimental results show, even 
though the constraint on pixel values is simple, effectively 
incorporating it in image restoration often improves the vis¬ 
ual quality and the quantitative results for a variety of im¬ 
ages and various restoration methods. Furthermore, for spe¬ 
cial classes of natural images that have distinct pixel values, 
it leads to considerable performance improvement. 


Kernel/Noise 

# of Images with PSNR Difference D 

> 0 

< 0 

> 0.01 

< -0.01 

33 X 33+3% 

129 

71 

24 

0 

45 X 45+2% 

126 

74 

22 

0 

51 X 51+1% 

132 

68 

68 

1 


Table 4: Quantitative comparison of TV+S and TV on 200 
BSDS500 images for the three kernel/noise settings. The PSNR 
difference is defined as D = PSNRtv+s — PSNRtv- 

5. Conclusion 

In this work, we describe a new regularizer to augment 
current image restoration algorithms when the original im¬ 
ages have known distinct pixel values. Our regularizer is in 
the form of a function and can be efficiently implemented 
with a soft-rounding operation. This regularizer can be 
readily incorporated into most existing image restoration 
methods and our experiments on restoration of binary text 
images, pattern images with multiple pixel values and natu¬ 
ral images show that its incorporation leads to considerable 
performance improvements to the state-of-the-art methods. 

There are a few directions we would like to further im¬ 
prove the current work. Currently, our method relies on the 
knowledge of the distinct pixel values in the original image. 
Another scenario is when we only know the exact number 
of distinct pixel values but not the values themselves. One 
subsequent study is to augment our algorithm so that it can 
also simultaneously estimate the distinct pixel values and 
restore the corrupted image. Furthermore, discriminative 
image restoration methods [21, 22] seem to be an effective 
alternative to the generative methods, and it is also of inter¬ 
est to combine our regularizer with such methods. 
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